Many researchers have suggested that local post-hoc explanation algorithms can be used to gain insights into the behavior of complex machine learning models. However, theoretical guarantees about such algorithms only exist for simple decision functions, and it is unclear whether and under which assumptions similar results might exist for complex models. In this paper, we introduce a general, learning-theory-based framework for what it means for an explanation to provide information about a decision function. We call an explanation informative if it serves to reduce the complexity of the space of plausible decision functions. With this approach, we show that many popular explanation algorithms are not informative when applied to complex decision functions, providing a rigorous mathematical rejection of the idea that it should be possible to explain any model. We then derive conditions under which different explanation algorithms become informative. These are often stronger than what one might expect. For example, gradient explanations and counterfactual explanations are non-informative with respect to the space of differentiable functions, and SHAP and anchor explanations are not informative with respect to the space of decision trees. Based on these results, we discuss how explanation algorithms can be modified to become informative. While the proposed analysis of explanation algorithms is mathematical, we argue that it holds strong implications for the practical applicability of these algorithms, particularly for auditing, regulation, and high-risk applications of AI.

It is only becoming more vital as in sensitive areas like Political Profiling, Review of Essays in AI gains foothold in making critical - and in some cases, Education, etc. proliferate, there is a great need for increasing fatal - decisions in sensitive areas like Healthcare, Finance, transparency in NLP models to build trust with stakeholders Automated Driving, and such-like [8] [9] [10]. The true potential and identify biases. A lot of work in Explainable AI has aimed to devise explanation methods that give humans insights into of these recent advancements in AI can only be realised the workings and predictions of NLP models. While these if the various stakeholders manage to discern the working of methods distill predictions from complex models like Neural AI models and how their predictions are produced, as that is Networks into consumable explanations, how humans understand necessary to incorporate trust. For example, 83% of people these explanations is still widely unexplored. Innate do not understand automated decision-making systems in the human tendencies and biases can handicap the understanding of these explanations in humans, and can also lead to them criminal justice system, and subsequently, 60% oppose its use misjudging models and predictions as a result. We designed in the domain [11]. But besides securing the buy-in of endusers a randomized survey-based experiment to understand the effectiveness and developers through building trust, AI explainability of saliency-based Post-hoc explainability methods also has the potential of identifying AI inaccuracies prior in Natural Language Processing.

We propose a simple objective evaluation measure for explanations of a complex black-box machine learning model. While most such model explanations have largely been evaluated via qualitative measures, such as how humans might qualitatively perceive the explanations, it is vital to also consider objective measures such as the one we propose in this paper. Our evaluation measure that we naturally call sensitivity is simple: it characterizes how an explanation changes as we vary the test input, and depending on how we measure these changes, and how we vary the input, we arrive at different notions of sensitivity. We also provide a calculus for deriving sensitivity of complex explanations in terms of that for simpler explanations, which thus allows an easy computation of sensitivities for yet to be proposed explanations. One advantage of an objective evaluation measure is that we can optimize the explanation with respect to the measure: we show that (1) any given explanation can be simply modified to improve its sensitivity with just a modest deviation from the original explanation, and (2) gradient based explanations of an adversarially trained network are less sensitive. Perhaps surprisingly, our experiments show that explanations optimized to have lower sensitivity can be more faithful to the model predictions.